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A method for visualising a spatially resolved data set using an illumination 
model 

..._) 

10 The invention relates to a method for visualising a spatially resolved data 
set using an illumination model as well as to the use of this method for 
the generation of three-dimensional representations of a body in accor- 
dance with the preamble of the independent claim of the respective cate- 

15 

For the representation of data sets of three or more dimensions, two- 
dimensional projections of these volumes or hypervolumes are often used 
which can then be graphically output and interpreted by the user. For the 
( quite frequent case that the volume is available in Cartesian or isotropic 

20 form, and the volumes are thus arranged in an orthogonal raster which 
has the same resolution in all three spatial directions, many graphics 
accelerators of modern data processing unit have hardware which can 
efficiently be used directly to carry out the volume visualisation. Such an 
equipping with graphics accelerators in the form of graphics cards has 

25 become an established standard in the meantime even with commercial 
personal computers. 
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The visualisation of spatially resolved data sets by three-dimensional 
pictorial representations, for example in one plane, is becoming increas- 
ingly important in many technical fields. This relates both to animations, 
for example for computer games or in advertising, and to the industrial 
5 sector and in particular to modern medical diagnosis and therapy. A 

number of image-producing examination methods are known here such 
as, among others, computer tomography, nuclear spin tomography or 
methods of ultrasonic technology, which should provide representations of 
^ specific regions of the human body, of organs, of the inside of blood ves- 

10 sels or of the heart, of the human skull, etc. It is here increasingly a ques- 
tion of imaging both substantially static images and moving processes in 
real time where possible. This is of central importance, for example in the 
observation of the movement of the heart by means of a catheter, or if a 
corresponding measuring probe, e.g. an ultrasonic probe, has to stand in 

15 for the eye of the physician in an operation. Related image-producing 
methods are naturally also well known from industrial engineering, for 
example for the non-destructive inspection of safety-relevant components 
such as wheelsets and axle sets in rail vehicles, pressure containers, pipes 
) and thin lines, e.g. in power plant technology and in many more other 

20 areas. 

Moreover, the visualisation of spatially resolved data sets by three- 
dimensional pictorial representations, in particular of radar data, sonar 
data (localisation and navigation) of seismic data sets, of weather data or, 
25 for example, the visualisation as part of finite element analyses is becom- 
ing more and more important. There are, however, also numerous applica- 
tions for computer simulations in the most varied areas, among others in 
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the area of radar engineering, ultrasound engineering and sonar engineer- 
ing. 

Importance is also increasingly being put on a more and more realistic 
representation of the dates sets detected in a technical measurement and 
then projected. This means that the trend is towards a higher and higher 
spatial resolution of the object to be observed, with it being a question of 
projecting the data set which is detected by a measuring apparatus and is 
as a rule three-dimensional in a perspectively correct manner and of 
inscribing light reflections into the projected volume in a realistic manner 
in order to assist human visual acuity in orientation in spatial depth in a 
three-dimensional graphical data set. The inscribing in of illumination 
effects into the projection is of particular importance in particular with 
stereoscopic projections, that is when a three-dimensional image should 
be communicated to the human visual acuity by a suitable projection 
device by superimposition of two projections slightly rotated in perspec- 
tive. 

) For this purpose, a specific illumination model is used as a base which is 

20 described by so-called illumination functions which approximate the 

interaction of light radiated in (virtually) with the objects in the volume, 
including attenuation, reflection and scattering. It is obvious that enor- 
mous computer power is necessary for this which cannot even easily be 
provided by the computer systems available on the market today. 

25 

The principle known from the prior art of the visualisation of multi- 
dimensional graphical data sets using commercial graphics hardware 
while taking a specific illumination model into account should be briefly 
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outlined in the following with reference to Figs. 1 to 3. To distinguish the 
prior art from the method in accordance with the invention, the reference 
numerals are provided with a dash in Figs, to 3. All the principles known 
from the prior art for the visualisation of data sets D', which include data 
D' (a*, P', y') (measured in a measurement coordination system K' m with 
coordinate axes a', P', y') have the fact in common that the illumination 
functions which define the illumination model are evaluated in the projec- 
tion space P, since the visual vector S' of the observer or of the illumina- 
tion vector is naturally defined there. Within the context of this applica- 
tion, the process of calculating the illumination functions is designated as 
"shading" on the basis of the nomenclature of the relevant literature. The 
process designated as "rendering" in the context of this invention must be 
distinguished from this. Rendering should be understood in the following 
as the linearisation after the cutting in an original volume or in a meas- 
ured data set and the subsequent transformation of the intersections into 
the geometry of a projection space. 

In a simple case, as shown schematically in Fig. 1 for a known example 
) from the prior art, the data set D' which is to be visualised and which was 

20 generated, for example, by an appropriate rule or by a measuring appara- 
tus 1' by a nuclear spin tomogram 1' by measurement in an original vol- 
ume Go', is available in Cartesian or isotropic form. The original volume 
Go' has the shape of a right parallelepiped and the volume elements V are 
arranged in an orthogonal raster which has the same resolution in all 
25 three spatial axes a', p', y' of the measurement coordinate system K' m . The 
data set D' is loaded into a data processing unit 3' for the projection of 
such a data set D', for example, onto a viewing monitor 2'. Said data 
processing unit has commercial graphics accelerators, which include 



5 



graphics hardware 4', which can be used efficiently to carry out the vol- 
ume visualisation. 

The concept of texture is frequently used for this, with the texture being 
5 defined by a data set from which two-dimensional polygonal surfaces can 
be copied, i.e. read, which corresponds to the aforesaid graphical cutting 
operation. 

For this purpose, as shown schematically in Fig. 2, any desired polygon is 
imaged onto another polygon E p ' in the projection space P' for the imaging 
of a digital image 5' from the original volume G 0 ', i.e. from a digital data 
set D', with the polygon Tp' of the original volume Go' not having to be 
geometrically similar to the polygon E p ' in the projection space P. If, as in 
the present example, an original volume G 0 ' is represented up by a three- 
dimensional data set D' which is built up of one or more 2D or 3D textures 
Tp', such a three-dimensional data set D' is frequently also termed a 3D 
texture. 

) It is understood that, in a particularly simple case, the texture Tp' can be 

20 identical to the polygon from the cutting operation. 

The original volume G 0 ', that is the data set D', is divided, for example 
perpendicular to a direction of observation B' of a (virtual) observer 6', into 
a specific number of textures Tp' whose corners are then rotated into the 
25 observation position, rotated, corrected perspectively and presented. This 
representation of the original volume Go* as a two-dimensional projection 
is achieved by repeated "cutting" of the original volume G 0 ', i.e. of the 
three-dimensional data set D' perpendicular to the direction of observation 
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6\ The cutting consists of the values being interpolated from the data set 
D' and projected into a projection plane which, in the present case, is 
identical to the projection space P\ 

The set of the slices which are two-dimensional images here must then be 
assembled to one single image 5' by an integration rule. The integration 
rule can be a simple point-by-point adding or can take place by a specific 
rule which is often also termed an illumination function as part of a so- 
called illumination model. The illumination functions or the illumination 
models therefore approximately take into account the interaction of light 
radiated in (really or virtually), including attenuation, reflection and scat- 
tering, with the objects of the original volume Go', that is the three- 
dimensional data set D', by the process of shading. 

15 If a stereoscopic projection should be achieved, the previously described 
method must additionally be carried out with respect to at least one sec- 
ond visual vector whose direction is slightly different from the visual vec- 
tor S* and must be supplied to a suitable stereoscopic projection device. 
) I.e., a view must be calculated for each eye in accordance with its position. 

20 

The literature of volume visualisation knows a number of approaches to 
ascribe optical features to a data set D' to be visualised. The best known 
class of such methods considers the intensity as a form of the optical 
density so that fluctuations in density cause light scattering and reflection 
25 and the density itself becomes light absorbing. In the literature, algo- 
rithms which are based on this hypothesis are generally termed gradient 
Tenderers. 
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The illumination functions are naturally evaluated in the projection space 
F, since the visual vector S' and the illumination vector are defined there. 
This means that the original volume Go 1 (that is the data set D') is initially 
cut into planes, i.e. sliced into textures Tp', and imaged in the projection 
space F, a process which is also termed texture remapping in the litera- 
ture. Subsequently, the illumination functions are evaluated in projection 
space F, that is are applied to the projected (that is rotated and corrected) 
textures Tp' in the projection space F. 

This method known from the prior art is disadvantageous, on the one 
hand, because the visual vector S' or the illumination vector do not have a 
constant amount either on the planar textures Tp' of the original volume 
Go' or on the projected planes E p ' in the projection space F; the amount of 
the visual vector is rather, as shown schematically in the left hand image 
of Fig. 3, constant at spherical shells K\ The visual vector S' and the 
illumination vector thus generally change over a slice, i.e. over a planar 
texture Tp' or over a plane E p ', in the projection space F. 

) The planes which are marked by visual vectors S' with a constant amount 

20 therefore form concentric spherical shells K' in the original volume G 0 ". 

Because amount- wise changes of the visual vector S' thus do not depend 
linearly on the texture coordinates of the textures Tp* or of the projected 
planes E p , these can also not be encoded directly on the graphics hard- 
ware 4'. In these methods known from the prior art, the projection plane 
25 Ep' must therefore be sliced into smaller areas in the projection space P' as 
shown schematically in the right hand image of Fig. 3, with linear interpo- 
lation being carried out between the corner points of the part areas. 
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In addition to various disadvantages such as an increase in the required 
calculation time, a particularly serious disadvantage lies in the fact that 
linearisation errors necessarily occur in the projection space P' due to the 
previously described interpolation. 

5 

Whereas the previously mentioned linearisation errors still move within a 
justifiable limit with the imaging of a substantially Cartesian and isotropic 
original volume G 0 \ that is of a Cartesian data set D f , the linearisation 
errors in non-Cartesian data sets D' are unjustifiably high so that either 

10 no usable image is produced in the projection space P' or the calculation 
effort increases so drastically that real time imaging such as is often abso- 
lutely necessary, for example in medical engineering, is no longer possible 
with the projection methods known from the prior art and with the cur- 
rently available hardware. It is understood that the calculation effort 

15 increases drastically again by an additional evaluation of the illumination 
functions in the projection space P' and is increased even further with a 
stereoscopic projection. 

The data sets D', which were acquired in the original volume G 0 ', are not, 
20 however, present in Cartesian, i.e. in orthogonal, coordinates in many 
applications important for practice. The reason for this is primarily the 
type of data acquisition. Typical examples are (X-ray) computer tomogra- 
phy or special ultrasonic techniques in which the data D* to be projected 
are present, for example, in cylindrical coordinates. When using very 
25 modern scanning systems, the relatively long time which is required for 
the preparation of a three-dimensional image in the projection space P' 
with the known methods of the prior art is caused less by the data acqui- 
sition as such, that is by the preparation of the data set D' per se, but 
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rather by the process of the visualisation of the data set D'. Ultrasonic 
probes are thus already known which scan very quickly and with which 
an original volume G 0 ' of interest is scanned in a plurality of planes simul- 
taneously and in circular form. Such ultrasonic probes include, for exam- 
5 pie, a plurality of ultrasonic converters which can be swivelled or rotated 
about an axis, or are arranged in linear form or generally in an array and 
of which some or all are operated in parallel so that a simultaneous cylin- 
drically symmetrical scanning is made possible simultaneously in a plu- 
rality of planes. Such fast scanning ultrasonic probes are very frequently 
10 not able to measure the volume in a Cartesian manner simply for reasons 
of efficiency. 

The data sets D' acquired in this way are thus present in cylindrical sym- 
metry and cannot be encoded directly on conventionally available graphics 
hardware. If a fast visualisation of the data sets D' is necessary, for exam- 
ple to achieve a real time representation at video frequencies with typical 
response times of less than 1/25 seconds, complicated and time-intensive 
calculating operations for the preparation of the measured data sets D' 
can basically not be allowed for the processing in the graphics hardware 
4'. In particular a complex coordinate transformation into a Cartesian 
coordinate system is thus eliminated. 

EP 1 059 612 recites a method for the visualisation of a spatially resolved 
data set D' to satisfy this problem which also allows non-Cartesian data 
25 sets D* to be processed enormously fast on generally available graphics 
hardware 4', such as is used in commercial personal computers, in a 
particularly efficient manner by avoiding complex part steps and so allows 
three-dimensional representations, even of moving objects, to be projected 
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and represented in real time, i.e. at typical video frequencies. The afore- 
said method is described in detail in EP 1 059 612 Al, whose content is 
herewith included in this application, and therefore no longer needs to be 
described in detail. 

5 

The method proposed in EP 1 059 612 A 1 admittedly allows multi- 
dimensional data sets D', which are present in any desired curvilinear 
coordinates, e.g. in cylindrical symmetry or spherical symmetry, to be 
encoded directly and enormously fast on conventional graphics hardware 
10 in a particularly elegant and efficient manner without any great calcula- 
tion effort. However, the problem of the fast and efficient evaluation of the 
illumination function remains unsolved. 

Starting from this prior art, it is therefore an object of the invention to 
15 provide a method for the visualisation of a spatially resolved data set that 
allows the illumination function to be evaluated in a particularly efficient 
manner, with the required calculation time being enormously reduced in 
comparison with methods of the prior art. 

20 The subject matters of the invention which satisfy this object are charac- 
terised by the features of the independent claim of the respective category. 

The dependent claims relate to particularly advantageous embodiments of 
the invention. 

25 

In accordance with the invention, a method is thus proposed for visualis- 
ing a spatially resolved data set using an illumination model, with one 
datum of the data set being respectively associated with one volume ele- 
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ment whose position is described by coordinates in a measurement coor- 
dinate system. The data are loaded into a graphics hardware as at least 
one texture in order to produce a pictorial representation in a projection 
space. The illumination model is evaluated in the measurement coordinate 
5 system. 

It is thus material to the invention for an illumination model used as the 
basis for the visualisation of the data set or for the illumination functions 
defining the illumination model to be evaluated in the measurement coor- 
dinate system. This means that the process of shading takes place com- 
pletely in the original volume and is completely separated from the process 
of rendering as it was initially described. 

The data of the data set which were generated in a measurement, for 
example in a measurement on the heart by means of an ultrasonic probe, 
are preferably processed without transformation from the measurement 
coordinate system into another coordinate system, in particular without 
transformation into a Cartesian and/or isotropic coordinate system. This 
means that when the method in accordance with the invention is used in 
which the illumination functions are evaluated in the measurement coor- 
dinate system, the data are prepared by elementary calculation operations 
such that they can be loaded into the graphics hardware and be processed 
by this. 

25 The method in accordance with the invention is preferably used in an 
embodiment in which the measurement coordinate system is a non- 
Cartesian measurement coordinate system. The measurement coordinate 
system can specifically be a cylindrical or spherical coordinate system or 
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another non-Cartesian coordinate system. The data can thus, for example, 
be generated by means of a rotating ultrasonic probe scanning at very 
high speed. Using such ultrasonic probes, it is possible to scan an original 
volume of interest, for example the interior of a human heart, simultane- 
ously and in circular form in a plurality of planes. Such ultrasonic probes 
can include a plurality of ultrasonic converters swivellable or rotatable 
about an axis and arranged in linear form or generally in an array such 
that a simultaneous cylindrically symmetrical scanning is made possible 
in a plurality of planes. The measurement coordinate system in which the 
data of the data set are present thus likewise has cylindrical symmetry. 

It is understood that the method in accordance with the invention can 
also advantageously be used in a special embodiment on data of a Carte- 
sian data set. The method in accordance with the invention is thus by no 
15 means restricted to non-Cartesian measurement coordinate systems. 

In an embodiment of the method in accordance with the invention which 
is important for practice, linear interpolation is carried out between the 
data of the data set in the measurement coordinate system. The data can 
thereby be loaded into the graphics hardware and be processed by this 
without carrying out coordinate transformation into a Cartesian system. 

Not least due to the fact that the data of the measurement data set which 
can be present in a curvilinear non-Cartesian measurement coordinate 
25 system, for example in cylindrical coordinates or in spherical coordinates, 
can be loaded into the graphics hardware and be processed by this with- 
out a previous coordinate transformation in an embodiment of the method 
in accordance with the invention which is very important for practice, the 
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data of the data set can also be evaluated close to a singularity. Such 
singularities can be found, for example, in cylindrical coordinates which 
can be described by a coordinate which corresponds to a radius, by an 
angular coordinate and by a further spatial coordinate at a value of the 
5 radius of zero. That is, the statement "a singularity is present at radius 
zero" is to be understood such that the data cannot be evaluated by a 
coordinate transformation in a measurement coordinate system with 
cylindrical symmetry in the proximity of or exactly at points having a value 
of the radius coordinate of zero. Since, however, no coordinate transforma- 

10 tion takes place in the embodiment described above, data of the data set 
close to a singularity can also be evaluated by application of this embodi- 
ment of the method in accordance with the invention. The method in 
accordance with the invention can naturally generally also be used when 
the data of the data set are subjected to a coordinate transformation, in 

15 particular for the further processing in the graphics hardware. 

In a further variant of the method in accordance with the invention, the 
data of the data set can in particular represent a volume-resolved scan of 
a body, for example of part of a human body such that the pictorial repre- 

20 sentation is a three-dimensional volume representation, in particular also, 
among other things, a semi-transparent representation of the body. Such 
representations are, among other things, of advantage when the observer 
has to orient himself in the volume of the body represented and/ or when 
the body or parts of the body represented is/are subject to specific mo- 

25 tions. The method can thus be used particularly advantageously, among 
other things, for the observation of a beating heart or in an operation on 
the beating heart, for example using an ultrasonic measuring device. 
Since it is possible to generate enormously fast three-dimensional repre- 
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sentations with the help of customary graphics hardware by using the 
method in accordance with the invention, three-dimensional images of 
very high resolution are possible in real time, also with video frequencies 
with response times of typically 1/25 seconds. 

5 

It is naturally also possible by using the method in accordance with the 
invention or by using one or more, i.e. by using a suitable combination, of 
the previously described embodiments, to generate the pictorial represen- 

^ tation as a stereoscopic projection. The inscription of illumination effects 

10 into the projection is of particular importance in particular for such 

stereoscopic projections, when - that is - the human visual acuity should 
be supplied with a three-dimensional image by super imposition of two 
projections slightly displaced in perspective by a suitable projection de- 
vice. Since the different embodiments of the method in accordance with 
15 the invention allow an enormously fast processing of the pictorial data, i.e. 
in particular allow an enormously fast and efficient processing of the 
illumination functions in the measurement coordinate system, even 
stereoscopic projections, possibly using suitable projection apparatuses 

) such as suitable 3D spectacles, even of movements, are possible in real 

20 time and at extremely high resolution. 

The method is this in particular suitable in its various embodiments for 
use for medical purposes, for the fast generation of three-dimensional 
representations of a body, in particular of a human body or parts thereof, 
25 using data gained by a technical measurement. 

It is understood that any suitable combination of the previously repre- 
sented embodiments can also be used advantageously and that the 
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method is furthermore exceptionally suitable not only for medical pur- 
poses, but very generally also in industrial engineering, e.g. for the inves- 
tigation of regions of a plant with difficult access. The data of the data set 
to be visualised in particular do not necessarily have to be generated by a 
5 technical measurement device such as an ultrasonic probe, but can also 
be made available, for example, by a mathematical rule, by a simulation or 
in a different manner. 

Before the invention is described in more detail with reference to the 
10 drawing, for the better understanding of the present application text, the 
mathematical principles material to the invention for the visualisation of a 
spatially resolved data set, in particular of a non-Cartesian data set, 
should be explained. 

15 Since, in the imaging process, planes of the original volume are not im- 
aged in planes, but - due to the perspective -in spherical shells in the 
projection space, the visual vector and the illumination vector change over 
a projected section. Because this change does not depend on the coordi- 
nates of the texture in a linear manner, but at best approximately, it 

20 cannot be encoded on the hardware as a texture operation. The texture is 
therefore divided into smaller part areas and is interpolated in a linear 
manner between the corner points of the part areas. The corner points of 
these part areas are also designated as vertices. Such a dividing into 
smaller part areas is designated as tessellation in the context of this appli- 

25 cation. The tessellation takes place in the original volume. The original 

volume is cut into concentric spherical shells such that the amount of the 
observation vector at a given texture which corresponds to a cut-out 
spherical shell remains constant such that the observation vector does not 
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have to be corrected. The resolution losses due to the correction opera- 
tions which usually occur are thereby minimised. The geometry is there- 
fore linearised in the original volume in that the illumination function is 
also evaluated in accordance with the invention. 

5 

The maximum error which arises due to a tessellation using angular 
increments of small angles a at an observation radius r is 
( (a\\ a 2 

^ err = r I- cos^-JJ « r—^- (eq. 1). An upper limit for the increment in mass 

1 (2 

units of pixels thus results at an error of err =< — as a < — , for r = 200, 

2 V r 

10 that is approximately six degrees. This error can be partly tolerated in 
Cartesian volumes, but is absolutely unjustifiably large in curvilinear 
coordinates such as in systems with spherical or cylindrical symmetry. 

One solution is the so-called "inner shading" in accordance with the pre- 
15 sent invention whose mathematical principles will be explained in more 
detail in the following. In inner shading, the geometry of the original vol- 
) ume is linearised in its non-Cartesian coordinates and the vertices are 

transformed into the geometry of the projection space. The observation 
vectors and illumination vectors are evaluated in the original volume and 
20 the illumination volumes are reshaped such that they apply in the original 
volume. The shading thus takes place completely in the original volume, 
from which the designation "inner shading " is derived. 
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The steps of shading and rendering are thus completely separated in 
accordance with the present invention. The original volume is first shaded 
at least element-wise and is then rendered via texture slices. If the total 
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original volume is first completely shaded and only then rendered, then 
one also speaks of a "two-stage inner shading" in this specific implementa- 
tion. 

5 To implement the method, the tessellation and the shape of the illumina- 
tion functions in the original volume must be given. The three coordinates 
in the original volume are designated by (a, p, y) and those of the Cartesian 
projection space as (x, y, z). The transformation T between the systems is 
^ (x, y, z) = T(oc, p, y) - T is La. not a linear operator; its inverse T- 1 must, 

10 however, exist, with the exception of the singularities. The corresponding 
linear operators are stated as follows: 

x - Tx(oc, p, y), y = T y (<x, p, y) and z = T z (oc, (J, y). 

15 In the case of spherical coordinates, (a, p, y) = (r, ((), cp) and 

x = r • co s(cp) cos (())), y = r • cos(cp)sin((|>) and z = r • sin(cp); one can use 
(a, p, y) = (r, <|>, z) for cylindrical coordinates, and obtains x = r -cos((j)), 
y = r • sin(cj)) and z = z. 

\ 
<■ 

20 It is known from the literature that a gradient can be converted into an 

illumination value in that it is used as an indicator in a data field which is 
arranged at the side faces of the unit cube. The values of the shading are 
therefore tabulated with reference to the vectors of the observation and of 
the illumination and are then stored in a table, termed a cube map. This 

25 reduces the problem to showing that the shading can be tabulated in 
dependence on the gradient in the original volume. 
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This is trivially the case - in the filling of the cube map, the gradient vector 
corresponds to the vector from the origin to the position in the cube map 
to be calculated; this vector n(0) is present at Go, it can therefore be trans- 
formed point-wise into the projection space by means of T: 
dT 



5 n (p)( x p) :=z Z — (*o) (eq. 2) 



dn 



M 



Its illumination function can then be calculated. 

dT 

The definition applies x p := T(xo), where — (x) is the direction derivation of 

dn 

dT 

10 T in the direction of n at the point x. Since both T and — are i.a. non- 

F dn 

linear, the cube map does not only have to be calculated again for each 
node of the tessellation, but it can also be incorrectly interpolated between 
the points. This general approach can therefore not be followed for curvi- 
linear coordinates. 

15 

It should be shown in the following that the shading values can be gained 
from the gradients of the original volume by means of the four elementary 
calculating operations. This is only conceivable as an approximation, 
which will be recited in the following, due to the generality of the trans- 
20 formation T. 

For this purpose, two vertices xo and x, of the tessellation are observed, 
between which all functions should be interpolated linearly. The Cartesian 
gradients g i := V(t(f(xi JJJ for each i can be recalculated into illumination 

25 values by hardware. However, only the gradients g 0l := v(p(x/)) are avail- 
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able which, in accordance with the invention, should not be transformed 
by a rotation. 

The normals must therefore be derived by the differential from T: a normal 
component g in the direction a at the coordinate jco = (a,p,y) is then ex- 
pressed in the projection space by g-^-ixo). Because the direction vectors 

da 9 

are locally defined and can be locally differentiated, they can be trans- 
formed linearly and the transition matrix Go — > G p (original volume by 
projection space) of the normal vectors at the point xo can be recited: 



10 N:= 



dT 

X 


dT x 


dT x 


da 


dp 


dy 


dT 

y 


dT y 


dT y 


da 


dp 


dy 


dT z 


dT z 


dT 2 


da 


dp 


dy 



This shows that the shading of non-Cartesian data 



sets can be carried out conventionally, but only after the transformation 
into the projection space. 
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The observations are therefore restricted in the following to a modification 

dT dT dT s 



15 of a base eG : = 



da 9 dp' dy 



of the projection space. N has the shape of 



the unit matrix at eG. The associated base eN := 



dT_ ^ 
dy 

\dfV ldf\ y II aril 



/ dT_ 
da 



dT 
dp 



I da I \dp\ \dy\ 



is or- 



thonormal when the original volume Go is locally orthogonal - which is the 
case with cylindrical and spherical coordinate systems - and can therefore 
be produced by rotation of the standard base of the projection volume G p . 
N in eN is a diagonal matrix with the elements {q a , qp, q Y } where 
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dT 

qa := fc~ 
\\da 





dT 




dT 








dy 



This proves that, for locally orthogonal 



systems, the gradient at Go corresponds to a Cartesian gradient, which 
was transformed by N- 1 . And N" 1 is the diagonal matrix with the recipro- 
cals of these q's. The gradient in Go ca therefore be used directly after it 
5 has been scaled component- wise with the reciprocals of the q's. A possible 
implementation of the shading at Go is thus already shown. 

However, even further properties of the shading should be exploited such 
that a more efficient implementation can be recited which also applies to 
10 non-orthogonal systems which have "drifting" base vectors. Such systems 
can be generated, for example, by so-called array scanners. Since the 
gradients themselves are never used in the prior art, but all illumination 

values are generated from the scalar products [g,l\[g,v) and (v,/) and from 
an optional scaling with ||g|| , possibilities for savings result here. 

15 

For this purpose, the equations only have to be brought into a suitable 

form in the original volume. The simple demand is then [gj)= [g 0i >Io) (eq. 

3) etc., that is the invariance of the scalar products. This demand can be 
trivially satisfied in that the same Euclidean scalar product is used which 
20 is independent of the base. This is, however, not a feasible path, because 
such a scalar product is complex in calculation. The criterion for the 
solution approach is simple: g should not be transformed, because a g 
arises for each pixel, but g and v are only evaluated in each vertex. 

\gjl) p = (g 0/ ,o(/o)) o (eq. 4) must therefore be made available with a standard 

25 scalar product for the original volume and the projection space which is 
independent of the base. 
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In the further procedure, 7o := s{lo) is explicitly derived. 

In the observed point x in the original volume Go, let the coordinates of the 
5 vectors be represented by the base e := ^a,ep,e r \, i.e. g := g a e a + g fi ep +g r e y 

and I := I a e a + I p ep + I y e r . If one observes the scalar products of the base 

c 0 := then cu = 1, but i.a. cy * 0 for the non-orthogonal bases. When 

written out, the scalar products read: 

fe,l)=g a I a +gpI fS +g Y l Y +gJp C a P +gJy C a Y + gp I a C aP + 8fi 1 y C P X + gy I a C ay + S r h C fir 

10 

(eq. 5), with only the first three terms existing with an orthonormal base e. 
If the expression in (eq. 5) is grouped by the components of g: 

(eq. 6) is recognized as a standard scalar product with a new vector V with 
the components 

20 S(l)=(l a (eq. 7). 

A universal transformation rule for the vectors I and v is thus available 
with S(I). No real geometrical significance may be ascribed to these new V 
and v', except that they keep the scalar products invariant in the original 
25 volume Go. 
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A very simple rule of minimum calculation effort is thus available which 
allows the calculation of the scalar products in the original volume Go in 
the same manner as in the Cartesian projection space, without T having to 
be evaluated for this purpose. The "c"s are in turn scalar products of the 
5 locals bases, will, however, be global constants for most systems - also for 
non-orthogonal systems - and therefore not consume any calculation 
effort. 

I and v are linearly interpolated at xo and x\ and therebetween by the 
10 graphics processor; the "g ,,r s, in contrast, are calculated at each pixel and 
not only at the vertices. The error introduced by the linearization likewise 
exists in the Cartesian rendering, since the visual vector sweeps over the 
visual field, but the sweeping is assumed as a linear operation. 

15 Nevertheless, due to the general projection T, the error of the interpolation 
is much more complex than in the Cartesian case: the illumination vector 
I and the observer vector v are both directional vectors which are gener- 
ated by norming at a point x: v := * * v lt (eq. 8) where x v is the observer 

position. We use (eq. 7) to obtain the simplest possible form of the direc- 
20 tional vectors. 

The same conditions therefore prevail at Go as in Cartesian systems. Since 
(eq. 3) is linear at Go, the non-linearity comes solely from the denominator 
of (eq. 8). The error of the linearisation by the reciprocal function can be 
25 estimated for a "sufficiently fine" tessellation in the sense that the de- 
nominator of (eq. 8) develops monotonically: 
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</):= Q 



r + (1-0- — 

d 0 d x _ 



1 



(eq. 9). t e [0..l] is the interpola- 



/•rf 0 +(l-r)tf,J 

tion parameter and the "d"s stand for the denominators in the vertices; 
and Q is a scaling which comes from the fact that the vectors transformed 
by (eq. 7) are no longer unit vectors. By the limitations of the "c"s in a3, 
Q e ]0..4[ applies. Since e(t)= 0,t e {0,l}, the error maximum can be evalu- 
ated at the extremes of e: 
*(')_ 



d Q -d l 



._ 0 J L + _ 

dt ~* d 0 d x + ( t .d 0 +(\-t) d,) 2 



= 0 (eq. 10a), which are evaluated, for 



the non-trivial case date do di (otherwise the error is identical to 0), to: 



io ( t (d 0 -dj+dj =- : f4=f4^. and *us 

_i L d \ ~ d o 



d 0 d l 



±Jd n d, —d, 

t= V ' (eq- 10b). 
d 0 -d x 



\ The solution is clear, since t e [0..l] and there must therefore be a plus sign 

15 before the root. 



This results in a handy criterion to determine the fineness of the tessella- 
tion. 
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By reinstating (eq. 10b) in (eq. 10a), one obtains: 
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(eq. 11). 



This gives the surprising result that, in a similar manner as for (eq. 1), 
errors<l can be forced. If do is predetermined, then (eq. 11) specifies in 
which range di can be selected. 



It is even the case that one can treat the tessellation comparatively care- 
lessly as long as one remains in the original volume Go, because one then 
does not approach any singularity and the "d"s remain comparatively 
large. The method is restricted by the geometric error of (eq. 1) and not by 
the errors of the shading approximation (eq. 9) for "d"s larger than 1. 

Previously the following was shown: the scalings and transformations 
necessary for the general shading and rendering can be carried out with 
elementary calculation operations. The correct illumination values are 
obtained in the vertices, the interpolated values contain at most errors of 
the second order as described in (eq. 9). The evaluation in the vertices is 
also elementary and the vertices are not very compact (and also do not 
have to be very compact) from the point of view of the calculation effort; 
their evaluation is consequently not the bottleneck of the calculation. 
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The invention will now be described in more detail in the following with 
reference to the schematic drawing. There are shown: 

Fig. 1 a method known from the prior art for visualising an original 

5 volume; 

Fig. 2 generation of a planar texture in accordance with the prior 

art; 

10 Fig. 3 linearising of a spherical shell-like texture; 

Fig. 4 an embodiment of the method in accordance with the inven- 

tion; 

15 Fig. 5 examples for textures in cylinder coordinates; 

Fig. 6 an embodiment for a linearisation in curvilinear coordinates. 



) Figures 1 to 3 show the prior art and were already discussed in detail in 

20 the above. 

Fig. 4 shows schematically the most important steps of an embodiment of 
the method in accordance with the invention. The important case for 
practice should be referred to as an example with reference to Fig. 4 that a 
25 data set D is based on measured values D(oc, p, y) which result from a 

volume-resolved scan of a body; and that a three-dimensional representa- 
tion is generated from this data set D. It is meant by "three-dimensional 
representation" that the representation actually is three-dimensional, or 
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that, for example, a stereoscopic projection is generated by a suitable 
projection apparatus, or that the projection takes place in a planar man- 
ner, for example on a computer monitor, but a three-dimensional impres- 
sion is provided, e.g. by means of methods of spatial or perspective repre- 
5 sentation, in particular using a corresponding illumination model. Such 
representations can specifically be semi-transparent such that they allow 
an insight into the scanned body. 

In the embodiment shown in Fig. 4, an original volume Go, for example the 
heart Go of a human body, is scanned with a measuring device 1 , in the 
present example with an ultrasonic measuring device 1 . Such an ultra- 
sonic measuring device 1 includes, for example, a plurality of ultrasonic 
converters 1 1 of which a plurality are arranged adjacent to one another 
with respect to an axis y. During operation, the ultrasonic measuring 
device 1 is rotated about the axis y, as is indicated by the double arrow at 
the axis y of the ultrasonic measuring apparatus 1 in Fig. 4. The individ- 
ual ultrasonic converters 1 1 are operated substantially in parallel such 
that the original volume Go, that is for example a heart, is scanned with 
ultrasound simultaneously in a plurality of parallel layers, which each lie 
substantially perpendicular to the y axis, over a sector (po X yo). The infor- 
mation with respect to the third dimension in the direction a is gained, for 
example, from the run time of the ultrasonic echo. 

The volume-resolved data D(ot, p, y) is thus present, optionally after a 
25 technical signal pre-processing, as a spatially resolved data set D which 
contains the information of the structure to be imaged. The dataset D is 
set up by so-called textures Tp which represent slices through the original 
volume Go- The measurement coordinate system K m , in which the data 
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D(oc, p, y) are present, is predetermined by the ultrasonic measuring device 
1 itself or by its manner of operation. The example described here is one of 
cylindrical coordinates. The textures Tp then correspond, as shown in Fig. 
5, to three different surface types, that is texture types Ta, Tp and Ty at 
5 which one each of the cylinder coordinates a, p, y has a constant value. 

The use of the method in accordance with the invention is naturally by no 
means restricted to cylindrical coordinates, but can also be used on other 
curvilinear coordinates and naturally also on Cartesian coordinates and 
what has been said above applies completely analogously for other coordi- 

10 nates than cylindrical coordinates. The data set D generated by the ultra- 
sonic measuring device 1 is loaded into a data processing unit 3 and is 
processed there. Optionally after a mathematical preparation phase, the 
illumination model, i.e. the illumination functions underlying this is 
evaluated first in the measurement coordination system K m , i.e. the "inner 

15 shading" described above is carried out. 

The data are then subjected to the "rendering" likewise described above 
and supplied to the graphics hardware 4 with whose aid a three- 
dimensional representation of the image 5 to be projected takes place, for 
example on an observation monitor 2. 

Fig. 5 shows schematically the three texture types for the case that the 
measurement coordinate system K m has cylindrical symmetry. If the 
measurement coordinate system K m has a different symmetry, for example 
25 spherical symmetry, the textures Ta, Tp, Ty or the surfaces representing 
them naturally have a different geometry corresponding to the symmetry 
of the coordinate system. 



) 
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In accordance with the invention, after the carrying out of the shading, i.e. 
after the evaluation of the illumination model BM in the measurement 
coordinate system K m , the data set D, or the textures Tp, are linearised in 
the measurement coordinate system K m , i.e. are subjected to the process 
5 of rendering. This should be briefly explained by way of example in Fig. 6 
in a schematic representation for a group of Ta textures of a data set in a 
cylindrically symmetrical measurement coordinate system K m . 

■J As already explained in detail, the visual vector S (or the illumination 

10 vector) will not have a constant amount either on planar textures Tp or on 
curvilinearly bounded textures Tp; it is rather the case that the amount of 
these vectors is constant on spherical shells K, as shown schematically in 
the left hand drawing of Fig. 6. The visual vector S thus generally changes 
over a slice, i.e. over a texture Tp. 

15 

Because changes in amount of the visual vector S as a rule thus do not 
depend linearly on the texture coordinates of the textures Tp, these also 
cannot be directly encoded on the graphic hardware 4. The curvilinearly 
) bounded textures Tp are therefore linearised, as shown in the middle 

20 drawing of Fig. 6, such that curvilinearly bounded part surfaces are cre- 
ated. The corner points of these part surfaces can now be encoded on the 
graphics hardware 4, as shown schematically in the right hand drawing of 
Fig. 6, and can be shown, for example, on an observation monitor 2 as 
Figure 5, without carrying out a coordinate transformation. 

25 

It is thus possible by the method in accordance with the invention to 
directly encode in a particularly elegant and efficient manner multi- 
dimensional data sets which are present in any desired curvilinear coordi- 
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nates, e.g. in cylindrical symmetry or in spherical symmetry, without any 
great calculating effort and enormously fast on customary graphics hard- 
ware while taking a corresponding illumination model into account. This 
is achieved in that the process of the so-called shading, that is of the 
evaluation of the illumination functions, and of the rendering, i.e. the 
process of the linearisation operations, are completely separated: the 
illumination functions are evaluated in the original volume, that is in the 
measurement coordinate system, and only then linearised. Since the 
evaluation of the data sets is possible with elementary calculation opera- 
tions and the shading takes place exclusively in the original volume and 
not only in the projection space, as known from the prior art, the method 
in accordance with the invention is enormously fast so that the visualisa- 
tion even of very complex data sets is possible in real time. Even the visu- 
alisation of moving processes in a stereoscopic representation becomes 
possible with high resolution and refresh rates which correspond to typi- 
cal video frequencies. Since the illumination functions are evaluated in the 
original volume and not in the projection space, it is possible with the 
method in accordance with the invention - thanks to its considerable 
speed in the carrying out of the visualisation operations - also to use very 
complex illumination models such that high resolution and realistic repre- 
sentations of previously unachieved quality can be achieved. 



